Field of the Invention
The present invention relates to a technique for assisting arrangement of an index.
Description of the Related Art
A mixed reality (MR) technique and an augmented reality (AR) technique are known as techniques by which the real world and the virtual world are merged. These techniques are techniques by which a physical space and a virtual space formed by a computer are merged seamlessly. These techniques are expected to be applied to various areas, such as assembly assistance in which work procedures and wiring are displayed in a superimposing manner at the time of assembly work, operation assistance in which the internal state of a patient is displayed in a superimposing manner on the body surface of the patient, and the like.
A big problem that needs to be solved in order to realize the MR technique is a problem of position alignment. In order for a user to feel that a virtual object is present in the physical space, geometric consistency between the virtual object and the physical space needs to be achieved. In other words, the virtual object needs to always be observed by the user so that it is located at the position at which it is supposed to be present in the physical space.
Also, a video see-through type information processing apparatus is an apparatus according to which an observer feels that the virtual object is present in the physical space. This is an apparatus that captures an image of the real world using a video camera, causes a display unit to display, on a display or the like in real time, a composite image obtained by superimposing the virtual object on the captured image, and provides the composite image to the observer. In general, examples of such information processing apparatuses include mobile information terminals known as tablet terminals having a video camera on the back, a video see-through HMD (head-mounted display), and the like.
With MR using the video see-through HMD, the position and orientation of a camera in the physical space at image capture is measured every time an image is input from the camera built in the HMD. In general, processing is performed in which CG is rendered based on the position and orientation of this camera, and a parameter unique to the camera, such as focal length, and the CG is superimposed on the image in the physical space. Therefore, in the case of MR using the video see-through HMD, the problem of position alignment is a problem relating to measurement of the position and orientation in physical space of the camera built in the HMD. In the past, there have been various research reports about measuring the position and orientation of the camera (Sato, Uchida, and Tamura: A Review of Registration Techniques in Mixed Reality. Journal of the VR society of Japan, 8(2): 171-180, 2003).
The position and orientation of the camera can be measured by, for example, a physical sensor having six degrees of freedom, such as a magnetic sensor, ultrasonic sensor, optical sensor, or the like. On the other hand, in the case where the video see-through HMD is used, it is possible to use image information from the camera built in the video see-through HMD for position alignment. The method for position alignment using image information has been widely used since it is simple and has a lower cost compared with the method using the physical sensor. With the position alignment method using image information, an index whose three-dimensional position in the physical space is known is captured by a camera to estimate the position and orientation of the camera based the correspondence between the position of the index on the captured image and the three-dimensional position. There are cases where markers manually arranged in the physical space, or natural features such as corner points and edges that are originally present in the physical space are used as indices.
There are cases in which the image features of indices in the image capture range cannot be sufficiently captured due to the position and orientation of the camera. For example, in the case where a person experiencing MR approaches CG so as to observe it, the indices are not in the angle of view of the camera, and thus the image cannot be captured. Also, in the case where the indices have a planar shape, if the image capture is performed in a direction orthogonal to the normal line of the plane, it is not possible to sufficiently capture the image of the indices. Therefore, it is important to arrange the indices such that the image features of the indices are sufficiently captured at any position and orientation of the camera. However, in the case where a plurality of indices are arranged, there is a possibility that the load for index detection will increase and index detection cannot be performed in real-time. Also, in MR where the physical space and CG are composited, inconveniences such as the beauty of a scene being ruined and the like occur.
In view of this, conventionally, indices have been arranged in accordance with guidelines derived from some kind of experience-based knowledge so that the indices are appropriately distributed and captured at an appropriate size. Furthermore, in the case where there are many indices, a design diagram that illustrates the location and type of indices that are to be arranged in the physical space is created.
With the technique described in Japanese Patent No. 4689380, indices that have already been arranged are used to present a location where a new index is to be arranged to a person arranging indices, so that appropriate indices can be arranged by a person experiencing MR. With the technique described in Japanese Patent No. 4689380, it is premised that precise position alignment can be performed using the indices that have already been arranged. A position at which a “precise position alignment region” that is determined using the already-arranged indices and a region where indices are to be arranged are overlapped seamlessly but not overlapped too much is superimposed on the physical space and presented as information for arranging indices that are to be newly added thereto.
With Japanese Patent No. 4689380, it is premised that precise position alignment is realized using prearranged indices, and it is an object thereof to add indices that are to be newly arranged. Thus, in a state before indices are arranged, it is not possible to present the position at which the indices are to be arranged.
Also, even if there is a design diagram for index arrangement created based on some kind of guideline, indices that are identified by a camera are designed to be easily identified by a computer, and therefore a person cannot easily identify them by just looking at the diagram. Therefore, there has been an issue in that it is difficult for a person to understand where indices are to be arranged in the physical space if he or she looks at only the printed design diagram or arrangement information displayed on a monitor.